The Six Assumptions (in order of importance)

1. Validity

The data you are analysing should map to the research question you are trying to answer.

2. Representativeness

The key assumption is that the data are representative of the distribution of the outcome \(y\) given the predictors \(x_1, x_2, \ldots\)

3. Additivity and Linearity

The deterministic component is a linear function of the separate predictors:

\[y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots\]

4. Independence of Errors

The simple regression model assumes that the errors from the prediction line are independent.

5. Equal Variance of Errors (Homoscedasticity)

Heteroscedasticity = unequal error variance

6. Normality of Errors

Why Plot?

Graphics are helpful for:

1. Visualising data - Understanding patterns
2. Understanding models - Seeing relationships
3. Revealing patterns not explained by fitted models

Displaying a Regression Line

# Simulated example
set.seed(853)
n <- 100
mom_iq <- rnorm(n, 100, 15)
kid_score <- 25 + 0.6 * mom_iq + 
  rnorm(n, 0, 18)

df <- data.frame(mom_iq, kid_score)

# Fit model
fit <- lm(kid_score ~ mom_iq, data = df)

# Plot
ggplot(df, aes(x = mom_iq, y = kid_score)) +
  geom_point(alpha = 0.6) +
  geom_smooth(method = "lm", se = TRUE) +
  labs(x = "Mother's IQ Score",
       y = "Child's Test Score") +
  theme_minimal(base_size = 14)

Displaying Uncertainty in the Fitted Regression

# Extract coefficients and simulate
coefs <- coef(fit)
se <- summary(fit)$coefficients[, 2]

# Create plot with uncertainty bands
ggplot(df, aes(x = mom_iq, y = kid_score)) +
  geom_point(alpha = 0.5) +
  # Add several simulated lines
  geom_abline(intercept = coefs[1] + rnorm(10, 0, se[1]),
              slope = coefs[2] + rnorm(10, 0, se[2]),
              alpha = 0.2, colour = "grey50") +
  geom_smooth(method = "lm", se = FALSE, 
              colour = "blue", linewidth = 1) +
  labs(x = "Mother's IQ Score",
       y = "Child's Test Score",
       title = "Regression with uncertainty") +
  theme_minimal(base_size = 14)

What Are Residuals?

The residuals are the differences between observed and predicted values:

\[r_i = y_i - \hat{y}_i = y_i - X_i\hat{\beta}\]

Creating a Residual Plot

# Calculate residuals and fitted values
df$fitted <- fitted(fit)
df$residuals <- residuals(fit)

# Residual plot
ggplot(df, aes(x = fitted, y = residuals)) +
  geom_point(alpha = 0.6) +
  geom_hline(yintercept = 0, 
             linetype = "dashed",
             colour = "red") +
  geom_hline(yintercept = c(-summary(fit)$sigma,
                            summary(fit)$sigma),
             linetype = "dotted",
             colour = "grey50") +
  labs(x = "Fitted Values",
       y = "Residuals",
       title = "Residual Plot") +
  theme_minimal(base_size = 14)

Interpreting Residual Plots

Residuals vs. Fitted vs. Observed

Common Residual Patterns

Why Transform Variables?

When additivity and linearity are violated, transformations can help:

Logarithmic Transformations

Interpreting Log-Scale Coefficients

For small coefficients (roughly \(|\beta| < 0.25\)):

\[e^\beta \approx 1 + \beta\]

# Verify the approximation
c(exp(0.06), 1 + 0.06)

[1] 1.061837 1.060000

Example: Earnings and Height

# Simulated earnings data
set.seed(123)
n <- 200
height <- rnorm(n, 170, 10)
earnings <- exp(6 + 0.02 * height + 
                rnorm(n, 0, 0.5))

df_earn <- data.frame(height, earnings)

# Compare models
fit_linear <- lm(earnings ~ height, 
                 data = df_earn)
fit_log <- lm(log(earnings) ~ height, 
              data = df_earn)

# Plot on log scale
ggplot(df_earn, aes(x = height, y = earnings)) +
  geom_point(alpha = 0.5) +
  scale_y_log10() +
  geom_smooth(method = "lm") +
  labs(x = "Height (cm)",
       y = "Earnings (log scale)") +
  theme_minimal(base_size = 14)

Natural Log vs. Log Base 10

Centering and Standardising

Centering: Subtract the mean \[x_{\text{centered}} = x - \bar{x}\]

Example: Centering for Interpretability

# Original model - intercept is meaningless
fit_orig <- lm(kid_score ~ mom_iq, data = df)
coef(fit_orig)

(Intercept)      mom_iq 
 25.7404791   0.5862104 

# Centered model - intercept is mean kid_score at mean mom_iq
df$mom_iq_c <- df$mom_iq - mean(df$mom_iq)
fit_centered <- lm(kid_score ~ mom_iq_c, data = df)
coef(fit_centered)

(Intercept)    mom_iq_c 
 83.2629997   0.5862104 

Square Root Transformation

Use when log transformation is too strong:

• Log: Equal ratios get equal treatment (5→10 same as 50→100)
• Square root: More moderate compression

# Compare transformations
x <- c(0, 100, 1000, 10000, 100000)
data.frame(
  original = x,
  log = round(log(x + 1), 2),  # +1 to handle zero
  sqrt = round(sqrt(x), 2)
)

  original   log   sqrt
1    0e+00  0.00   0.00
2    1e+02  4.62  10.00
3    1e+03  6.91  31.62
4    1e+04  9.21 100.00
5    1e+05 11.51 316.23

The Mortality Rate Example

Aggregation Bias Explained

During 1999-2013:

• The average age within the 45-54 group increased
• Baby boomers were moving through this age bracket
• Older people within this bracket have higher mortality rates

What Proper Adjustment Revealed

After adjusting for age composition:

• The steady increase from 1999-2013 disappeared
• Instead: increase from 1999-2005, then constant thereafter
• Breaking down by sex: marked increase only for women, not men

Why Tables Matter

Tables can communicate specific values with high fidelity:

1. Show an extract of the dataset
2. Communicate summary statistics
3. Display regression results

Showing Data with kable()

# Basic table with kable
df |>
  head(5) |>
  select(mom_iq, kid_score, fitted, residuals) |>
  kable(
    col.names = c("Mother's IQ", "Child Score", 
                  "Fitted", "Residuals"),
    digits = 1,
    caption = "First 5 observations from the dataset"
  )

Summary Statistics with modelsummary

df |>
  select(mom_iq, kid_score) |>
  datasummary_skim(
    histogram = FALSE,
    title = "Summary statistics for mother-child data"
  )

Regression Tables with modelsummary

# Fit multiple models
model1 <- lm(kid_score ~ mom_iq, data = df)
model2 <- lm(kid_score ~ mom_iq + I(mom_iq^2), data = df)

# Display comparison
modelsummary(
  list("Linear" = model1, "Quadratic" = model2),
  fmt = 2,
  title = "Comparing linear and quadratic models"
)

Table Best Practices

The Process of Writing

Paper Structure

A quantitative paper typically includes:

Writing Effective Abstracts

An abstract should cover (in ~4-5 sentences):

1. Context - The general area and why it matters
2. Objective - What you’re doing
3. Approach - Data and methods
4. Findings - The headline result
5. Implications - Why it matters

The Data Section

Rules for Good Writing

1. Focus on the reader and their needs
2. Establish a structure and stick to it
3. Write a first draft quickly
4. Rewrite extensively
5. Be concise - remove unnecessary words
6. Use words precisely
7. Avoid jargon

Residual Standard Deviation (\(\sigma\))

\[\hat{\sigma} = \sqrt{\frac{\sum_{i=1}^{n}(y_i - \hat{y}_i)^2}{n-k}}\]

# Get sigma from our model
summary(fit)$sigma

[1] 15.25318

R-squared (\(R^2\))

The proportion of variance “explained” by the model:

\[R^2 = 1 - \frac{\hat{\sigma}^2}{s_y^2}\]

summary(fit)$r.squared

[1] 0.2243996

Cross-Validation

Problem: Using the same data to fit and evaluate leads to optimism.

Comparing Models

# Add a noise predictor
set.seed(456)
df$noise <- rnorm(nrow(df))

model_simple <- lm(kid_score ~ mom_iq, data = df)
model_noise <- lm(kid_score ~ mom_iq + noise, data = df)

# Compare R-squared
c(simple = summary(model_simple)$r.squared,
  with_noise = summary(model_noise)$r.squared)

    simple with_noise 
 0.2243996  0.2268193 

Diagnostic Workflow Summary

Key R Functions

Summary

Next Week

Week 9: Logistic Regression

• Modelling binary outcomes
• Odds ratios and log-odds
• Making probabilistic predictions